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ABSTRACT 


Thu  paper  introduces  a  new  approach  to  the  problem  erf  information  proemsing  in  autonomous  undersea  vehicles  (AUVs).  The 
approach  involves  the  emulation  of  biological  mechanisms  of  intelligence,  principally  through  the  evolutionary  design  of  neural  net¬ 
works  and  systems  of  interacting  networks,  for  implementation  in  software,  parallel  hardware,  and  analog/  digital  very -large-scale  inte¬ 
gration  (VLSI).  The  objectives  of  artificial  neural  systems  (ANS),  in  common  with  other  AI  approaches,  are  to  provide  real-time, 
operator-independent,  pattern  recognition,  task  planning,  and  adaptive  AUV  control  for  many  of  the  underwater  tasks  currently  being 
performed  by  man.  The  ANS  approach  emphasizes  the  development  of  autonomous  intelligence  capabilities  through  the  progressive 
expansion  of  processing  centers  that  define,  orchestrate,  and  control  basic  reflexes. 

An  example  of  this  approach  is  our  current  IED  project  in  machine  visual  motion  analysis.  The  perception  of  motion  is  well 
developed  in  animals  from  insects  to  man,  providing  critical  information  for  obstacle  avoidance  and  predator  and  prey  recognition.  We 
have  therefore  applied  a  model  of  the  motion  analysis  subnetwork  in  the  vertebrate  retina  to  visual  navigation  in  the  AUV.  The  model 
is  currently  implemented  in  the  C  programming  language  as  a  discrete-time  serial  approximation  of  a  continuous-time  parallel  process. 
Running  on  an  IBM-PC /  AT  with  digitized  video  camera  images,  the  system  can  detect  and  describe  motion  in  a  16-  by  16-receptor 
field  at  the  rate  of  4  updates  per  second.  The  system  responds  accurately  to  images  with  different  speeds  and  directions  at  signal-to- 
noise  ratios  greater  than  three.  The  architecture  is  quite  simple  and  thus  appropriate  for  implementation  in  VLSI  optoelectronics. 

INTRODUCTION 


PROBLEM 


Within  the  next  20  years,  the  Navy  will  need  a  new  class  of  vehicles  that  are  small,  inexpensive,  reliable,  and  capable  of  extended 
missions  without  human  control,  supervision,  or  intervention.  These  autonomous  underwater  vehicles  (AUVs)  will  find  significant  em¬ 
ployment  in  undersea  surveillance,  deep-sea  exploration,  salvage,  and  mine  countermeasures.  These  applications  are  well  recognized, 
and  work  at  the  Naval  Ocean  Systems  Center  (NOSC)  over  the  last  decade  has  progressed  toward  their  achievement.  Products  include 
tethered  and  teleoperated  submersibies,  semiautonomous  vehicles  with  on-board  computers  for  navigation,  pattern  recognition,  and 
vehicle  control,  fiber-optic  data  links,  and  untethered  acoustic  links  supporting  high-level  communication. 


Figure  I  shows  three  representative  NOSC  undersea  vehicles.  Tethered  submersibies  like  the  Mine  Neutralization  System  (MNS) 
have  provided  remote  operation  capability,  but  suffer  from  the  drag  and  short  range  of  an  umbilical  cable.  On-board  energy  supply,  a 
supervisory-control  software  architecture,  and  single-strand  fiber-optic  data  link  increased  the  freedom  of  movement  and  range  for  the 
Experimental  Autonomous  Vehicle  (EAVE-WEST).  Acoustic  telemetry  was  developed  for  the  Advanced  Unmanned  Search  System 
(AUSS)  program  which  cut  the  physical  link  altogether.  The  loss  of  the  hard  wire  umbilical  cables  in  these  two  later  systems,  however, 
forced  the  vehicles  to  carry  their  own  energy  supply,  thus  limiting  the  duration  of  work. 

In  spite  of  the  on-board  computers,  the  submersibies  still  could  not  function  independently  of  supervisory  control,  and  that  con¬ 
trol  was  itself  labor  intensive.  The  von  Neumann  computer  architectures  are  ill-suited  to  the  parallel  data  processing  requirements  asso¬ 
ciated  with  pattern  recognition,  data  fusion,  task  planning  and  adaptive  control,  and  the  number  of  lines  of  software  code  being  written 
to  address  those  issues  face  a  combinatorial  explosion.  Thus,  there  remain  two  major  problems  that  stand  in  the  way  of  the  realization 
of  true  autonomous  submersibies.  One  is  an  energy  source  or  renewal  strategy  that  will  sustain  long-term  voyages;  the  other  is  an  effec¬ 
tive  on-board  computer  that  will  replace  the  control,  pattern  recognition,  and  task  planning  functions  of  human  operators.  This  paper 
will  focus  on  a  new  approach  to  the  problem  of  on-board  computation. 


APPROACH 


Our  effort  to  develop  AUVs  is  in  many  ways  analogous  to  natural  evolution.  We  are  fortunate  to  be  operating  in  a  relatively 
uncomplicated  environment.  The  ocean  is  homogeneous,  there  is  little  to  bump  into,  and  those  objects  which  are  moving  are  moving 
slowly.  Our  performance  requirements  are  similarly  simple.  Current  vehicles  are  asked  only  to  track,  to  find,  to  fetch,  or  to  avoid. 
These  abilities  are  present  in  some  of  the  simplest  marine  invertebrates  (species  lacking  a  backbone),  which  to  the  same  degree  are 
autonomous.  Yet  comparable  abilities  have  not  been  attained  by  any  class  of  autonomous  machine  using  any  technology  available  to 
date.  Our  approach  therefore  is  to  examine  the  methods  used  by  marine  species  over  the  course  of  phylogenetic  development  to  meet 
their  evolving  behavioral  requirements.  A  judicious  emulation  of  these  methods,  also  using  an  evolutionary  approach,  should  provide  a 
good  foundation  for  the  types  of  artificial  systems  that  are  required  for  Navy  missions. 

There  is  a  rich  source  of  information  available  from  the  fields  of  neuroethology  and  oehavioral  neurobiology  on  the  behavioral 
competence  of  marine  invertebrates  and  some  of  the  neural  networks  and  mechanisms  that  generate  those  adaptive  behaviors.1  <2,3,4 
The  fourth  column  of  Table  I  lists  the  increasing  complexity  of  behavior  across  a  number  of  species,  while  the  third  column  lists  new 
features  of  the  nervous  system  that  likely  support  the  advances  in  capability.  Note  that  at  the  various  levels  of  the  phylogenetic  scale 
one  can  find  homing,  tracking,  predation,  escape  and  evasion,  migration,  pattern  recognition,  learning,  and  memory.  Alt  ate  desirable 
functions  for  autonomous  submersibies. 
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It  is  instructive  to  examine  how  nature  has  developed  these  capabilities.  First,  basic  reflexes  were  established  to  orient  the  animal 
to  critical  features  in  its  environment,  such  as  light,  gravity,  and  sources  of  energy.  These  basic  reflexes  can  be  found  in  all  nervous  sys¬ 
tems  from  the  jellyfish  to  man,  and  provide  the  foundation  for  all  more  complicated  behaviors.  The  organized  species-specific  behaviors 
such  as  mating  and  nest  building,  and  the  acquired  behaviors  such  as  language  and  art.  are  produced  through  the  modulation  and 
coordination  of  existing  innate  reflexes.  Without  the  basic  reflexes,  the  more  complicated  behavior  would  not  be  possible.  Reflexes  also 
function  semi-independently  to  regulate  internal  conditions,  correct  for  gravity  and  other  unexpected  forces,  and  avoid  hazards.  These 
reflexes  can  maintain  life-support  functions  when  the  central  controllers  are  distracted  or  disabled  an  excellent  design  feature  for  a 
battle-hardened  system. 

PROSPECTS  FOR  AN  ARTIFICIAL  NEURAL  SYSTEM 

A  major  difficulty  of  the  artificial  neural  systems  (ANS)  approach  is  the  lack  of  an  appropriate  hardware  for  implementation. 
While  much  work  is  going  on  in  this  area,3  there  are  no  generally  available  neural  computers  that  run  networks  in  parallel  and  do  not 
consult  a  program  of  stored  instructions. 

Asking  for  such  a  computer  may  be  the  wrong  approach.  The  success  of  biological  neural  networks  stems  from  the  appropriate¬ 
ness  of  their  design  to  the  environment,  and  to  the  sensor  and  effector  equipment  of  their  species.  The  common  design  features  shared 
by  most  biological  neural  systems  can  be  attributed  to  a  common  environment  and  to  simitar  equipment.  When  more  processing  was 
required  to  survive  in  a  varied  environment,  the  earlier  design  was  not  abandoned;  it  was  simply  expanded.  An  increasingly  complex 
biological  neural  system  was  developed  by  building  hierarchical  control  and  integration  systems  on  existing  more  primitive  compo¬ 
nents.  AH  biological  neural  systems  arc  highly  organized  assemblies  of  functionally  differentiated  components,  just  as  a  piece  of  elec¬ 
tronic  equipment  is  assembled  from  discrete  components.  These  components,  however,  when  modeled  after  biological  neural  networks, 
would  manifest  well-known  advantages  of  fault  tolerance,  ambiguity  tolerance,  speed,  and  adaptability  that  are  consequences  of  dis¬ 
tributed  parallel  procesang.  An  artificial  neural  computer  should  be  similarly  constructed.  Instead  of  an  unstructured  collection  of 
homogeneous  processing  elements,  to  be  rewired  for  each  different  application,  the  neural  computer  should  be  assembled  from  tuned 
circuits.  These  circuits  should  have  a  functional  specificity,  but  alto  have  a  certain  degree  of  interchangeability  from  one  neural  com¬ 
puter  to  the  next.  Why  this  hat  not  happened  can  be  attributed  to  the  lack  of  good  functional  components.  There  are  no  off-the-shelf 
analogues  of  the  retina  (early  vision),  optic  tectum  (visual  motor  control),  hypothalamus  (Mental  state  monitor  and  controller),  or 
locus  coetilm  (arousal).  Our  program  is  an  effort  to  provide  those  components  and  assemble  them  into  working  control  systems.  An 
application  of  this  approach  to  visual  motion  analysis  for  AUVs  will  be  described  next. 
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MODEL  OF  MOTION  ANALYSIS  IN  VERTEBRATE  RF’HNA 


The  purpose  of  machine  vision  is  to  provide  information  on  the  relative  location,  change  in  location  or  movement,  and  distin¬ 
guishing  characteristics  of  objects  in  the  visual  field.  This  information  will  be  used  by  a  vehicle  in  the  performance  of  some  physical 
work,  such  as  navigation,  search  and  recovery,  or  repair.  To  do  so,  the  visual  system  must  provide  input  and  feedback  to  vehicle  effec¬ 
tor  controllers.  Additionally,  autonomy  dictates  that  the  visual  processing  must  take  place  on  the  vehicle,  and  a  free-swimming  vehicle 
requires  that  the  visual  processing  take  place  in  real  time.  To  date,  there  have  been  few  successes  in  the  development  of  an  on-board 
machine  vision  system  that  provides  real-time  analysis  of  motion  for  a  moving  vehicle 

lr  contrast  to  our  difficulties  with  machine  vision,  most  marine  animals  have  solutions  to  vision  requirements  under  water. 

These  r  .ige  from  light-sensitive  spots  that  allow  orienting,  to  complex  retinal  and  cortical  processing  of  form,  color,  and  motion.  The 
compound  eyes  of  arthropods,  including  flies,  beetles,  lobsters,  and  crabs,  are  designed  primarily  to  detect  motion,6  and  while  the  ver¬ 
tebrate  visual  system  has  advanced  pattern  analysis  across  the  phylogenetic  scale,  it  has  remained  extremely  sensitive  to  movement.7 

MOTION  ANALYSIS  IN  THE  VERTEBRATE  RETINA 

The  neural  connectivity  of  the  vertebrate  retina  has  been  studied  extensively,®'®’ l®»l  *  so  there  is  much  to  draw  upon  in  the  con¬ 
struction  of  a  mo' el.  It  appears  that  visual  systems  are  organized  to  perform  two  complementary  visual  processes.  One  involves  pattern 
and  color,  the  otl.:r  involves  motion  and  spatial  relationships.  I  M2  The  neural  mechanisms  of  these  two  basic  processes  are  evidently 
distinct  in  the  retinal  architecture  and  remain  separable  far  into  the  cerebral  cortex  J  2  Interactions  of  these  two  processes  are  likely,  but 
?r-  unnecessary  for  the  level  of  performance  required  of  the  present  model. 

Pattern  and  color  analysis  in  the  vertebrate  retina  is  initiated  by  a  network  involving  cone  receptors  and  sustained-activity 
X-type  ganglion  cells.  Motion  analysis  begins  in  the  retina  with  a  network  involving  rod  receptors  and  transient-activity  Y-type  gan¬ 
glion  cells. 14  While  the  cone/  X-type  ganglion  cell  circuit  exists  side  by  side  with  the  rod/  Y-type  ganglion  cell  circuit,  the  fact  that  the 
two  types  of  image  processing  networks  are  distinct  eases  the  degree  of  retinal  complexity  to  which  we  must  attend.  Our  model  thus 
focuses  on  the  rod  to  Y-ganglion  cell  subnetwork  for  motion  analysis. 

Between  the  rods  and  Y-ganglion  cells  of  the  motion  analysis  subnetwork  are  beta  horizontals,  rod  bipolars,  and  more  than  one 
type  of  amacrinc  cell.  U  This  subnetwork  is  shown  schematically  in  Fig.  2.  The  beta  horizontals  receive  input  from  several  rod  recep¬ 
tors1 3  and  inhibit  rod  bipolars. 16  The  rod  bipolars  communicate  with  the  two  types  of  amacrinc  cells.  The  first  type  is  a  low  density 
but  large  diameter  receptive  field  cholinergic  amacrinc  (A-I)  interposed  between  the  rod  bipolar  and  Y-ganglion  cells.17  The  second 
type  of  amacrinc  (A-II)  has  a  higher  density  in  the  retina  but  each  cell  has  a  smaller  diameter  receptive  field  than  the  A-l  amacrines. 
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Figure  2.  Biological  model  of  motion  anaJym  sub¬ 
network  in  vertebrate  retina. 
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The  neurotransmitter  of  the  A-II  amacrine  is  likely  gama-a miqobutyric  acid  (GABA),  a  known  inhibitor.  Pharmacological  blockade  of 
this  neurotransmitter  by  picrotoxin  was  found  to  eliminate  the  null  response  of  direction  selectivity  (defined  in  the  next  paragraph)  in 
the  rabbit  retina.  **  Rod  bipolars  converge  on  the  A-U  amacrines  with  a  ratio  of  about  30:1.  It  is  posable  that  the  A-il  amacrines 
inhibit  the  Y-ganglion  cells,  while  the  A-II  amacrines  themselves  may  be  under  inhibitory  control  from  the  rod  bipolars.  *9  JO  Masland, 
Mills,  and  Cassidy?  1  suggest  that  despite  the  large  dendritic  field  of  cholinergic  (A-I)  amacrine  cells,  their  output  is  localized  to  the 
immediate  region  surrounding  the  bipolar  input.  Since  the  dendrites  of  the  ganglion  cells  extend  into  the  inner  plexiform  layer  and 
intermingle  with  bipolar  axon  arborizations  and  amacrine  dendritic  arborizations,  both  types  of  amacrine  cells  have  the  opportunity  for 
local  control  over  bipolar  to  ganglion  transmission. 

DIRECTION  SELECTIVITY 

Direction  selectivity  has  been  recorded  from  retinal  ganglion  cells  in  several  different  species,  including  goldfish,  frog  and  toad, 
turtle,  pigeon,  cat  and  rabbit.??  Barlow  and  Hill?^  reported  that  some  ganglion  cells  in  the  rabbit  responded  vigorously  when  succes¬ 
sive  stimuli  were  presented  in  one  direction  across  their  receptive  fields  (the  preferred  direction),  and  decreased  their  activity  when 
movement  was  in  the  opposite  direction  (the  null  direction  for  that  ganglion  cell).  The  receptive  field  is  that  region  of  the  visual  field  to 
which  the  ganglion  cell  responds.  It  is  generally  concentric  -  encompassing  up  to  8  degrees  of  visual  angle.?4  Barlow  et  al.?4>?5  pro¬ 
posed  that  direction  selectivity  was  the  result  of  an  inhibitory  mechanism  that  vetoed  responses  in  the  null  direction. 

Current  thought  on  the  mechanism  for  direction  selectivity  in  the  vertebrate  retina  (shown  in  Fig.  3)  remains  based  on  the  con¬ 
vergence  of  excitation  and  inhibition  at  the  input  to  the  ganglion  dendrite.  I  *  >2?  The  functional  components  of  the  mechanism  involve 
only  the  two  different  types  of  amacrine  cells:  one  inhibiting,  and  the  other  exciting  the  direction-sensitive  ganglion  cells.  The  two  types 
are  arranged  spatially  so  that  a  moving  image  activates  first  one  then  the  other.  The  preferred  direction  requires  activation  of  excitation 
in  advance  of  inhibition,  while  the  null  direction  requires  the  converse. 
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Figure  3.  Mechanism  of  direction  selectivity. 
Images  moving  from  left  to  right  generate 
excitation  in  advance  of  inhibition  at  the 
ganglion  cell. 


SYNOPSIS  OF  THE  MODEL 

There  is  more  than  one  way  to  connect  processing  elements  in  a  model  retina  to  achieve  direction  selectivity.  In  this  paper,  we 
will  describe  and  contrast  two  of  our  more  successful  variations.  One  method  involves  inhibition  of  bipolar  to  ganglion  transmission 
(I-va riant),  the  other  involves  facilitation  of  bipolar  to  ganglion  transmission  (F-va riant). 

The  basic  retinal  model  for  motion  processing  is  a  feed-forward  sparsely  interconnected  neural  network,  based  on  the  models 
shown  in  Figs.  2  and  3.  The  connection  patterns  for  the  1  and  F  variations  of  this  network  are  shown  in  Figs.  4a  and  4b  respectively, 
and  described  below.  The  numbers  of  elements  in  the  networks  can  be  arbitrary,  dependent  only  on  the  size  and  resolution  of  the  dig¬ 
itized  image,  and  on  the  ability  of  the  computing  machine  to  process  the  information  in  the  required  time.  However,  the  numbers  of  the 
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Figure  4.  Two  variations  on  the  mechanism  of  direction  selectivity. 

(a)  Show*  the  use  of  persistent  lateral  inhibition  to  block  conduction 
in  the  null  direction,  (b)  Shows  the  use  of  persistent  lateral  excitation 
to  facilitate  conduction  in  the  preferred  direction. 


different  types  of  elements  maintain  fixed  proportions  to  the  total.  There  are  six  different  types  of  elements,  roughly  analogous  to 
receptor,  horizontal,  bipolar,  amacrine  I  and  II,  and  ganglion  cells  of  the  biological  retina.  Elements  of  like  type  are  assembled  into  lay¬ 
ers.  The  layers  are  connected  by  delay  lines.  Activity  originating  in  the  receptors  percolates  through  the  retina,  delayed  by  one  time 
constant  (t)  at  some  connections  as  noted  below.  The  time  constant  in  the  current  implementation  is  defined  by  the  program  cycle  time. 

The  receptive  field  is  divided  into  sectors  by  the  convergence  of  an  area  of  bipolars  on  A-l  amacrines  and  of  A-II  amacrincs  on 
ganglion  elements.  As  a  minimum,  we  use  four  sectors  which  divide  the  visual  field  into  four  equal  quadrants. 

Receptor  Elements.  The  receptor  (R)  matrix  is  represented  by  gray-scale  values  which  can  originate  from  an  image  simulator  or 
a  frame  grabber  that  has  digitized  television  camera  images  or  a  matrix  from  some  other  sensor.  Simulator  and  frame  grabber  values 
range  between  0  and  2S6.  Higher  numbers  are  associated  with  brighter  light.  These  values  are  forwarded  to  horizontal  and  bipolar 
matrices  in  columnar  pathways. 

Horizontal  Elements.  Horizontal  (H)  elements  receive  input  without  delay  from  the  nearest  receptor  located  immediately  above. 

Hijftr  Rij(t) 

where  i  indexes  the  position  on  the  X  axis,  and  j  indexes  the  position  on  the  Y  axis  of  a  Cartesian  map  of  the  receptor  surface.  Horiz¬ 
ontal  element  output  is  forwarded  to  the  bipolar  element  located  immediately  below,  after  a  delay  of  one  cycle. 

Bipolar  Elements.  Bipolar  (B)  elements  compare  the  inputs  from  local  receptor  and  horizontal  dements,  passing  on  the  positive 
value  of  the  difference.  The  choice  of  differencing  is  not  critical  for  the  performance  of  the  model.  Bipolar  output  coual  be  made  con¬ 
tingent  upon  only  increases  in  light  intensity  (R-H),  upon  only  decreases  in  light  intensity  (H-R),  or  upon  change  in  either  direction, 
derived  by  taking  the  absolute  value  of  the  difference  (|R-H|). 
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The  local  change  in  light  intensity  is  transferred  to  the  two  types  of  amaenne  elements.  The  bipolar  output  is  passed  to  the  A-l  ama- 
critic  element  in  its  local  sector  and  also  to  the  A-II  amacrine  elements  through  a  connection  matrix  that  determines  sensitivity  to  direc¬ 
tion  of  intensity  change  actoss  the  receptor  layer. 


Amacrine  Elements.  Type  A-l  amacrine  (Al)  elements  integrate  input  only  from  bipolar  elements  in  the  local  sector  and  send 
output  to  adjust  thresholds  of  sector  ganglion  elements.  A  percentage  (g)  of  A-l  activity  persists. 

AI(t)  =  I  *  A1(t-I)  *  2  Bij(t) 0M 

It  is  the  connectivity  and  functions  of  the  A-II  amacrines  that  contribute  to  the  variation  in  our  model  of  direction  selectivity.  In 
the  l-variant,  there  is  persistent  inhibition  in  the  null  direction.  Bipolar  input  accumulates  and  persists  in  A-II  amacrine  elements  which 
then  inhibit  and  block  bipolar  conduction  to  ganglion  elements  in  the  null  direction.  In  the  l-variant,  each  ganglion  element  is  asso¬ 
ciated  with  a  unique  field  of  A-ll  amacrine  elements  (or  dendritic  regions  of  a  few  large  amacrine  elements)  that  store  the  inhibition.  In 
the  F-variant,  persistent  excitation  in  the  preferred  direction  is  used.  There  is  only  one  field  of  amacrine  elements  and  this  is  shared  by 
each  of  the  four  direction-sensitive  ganglion  elements  in  a  sector.  Excitation  is  distributed  and  stored  horizontally  in  advance  of  move¬ 
ment  in  the  preferred  direction  and  used  to  facilitate  the  bipolar  to  ganglion  transmission.  The  differences  in  the  two  models  will  now 
be  described  separately,  beginning  with  the  model  using  persistent  inhibition. 


Type  A-II  amacrine  (All)  elements  sum  bipolar  output  within  small  receptive  fields  of  a  sector  with  any  residual  inhibition  (with 
a  persistence  factor  of  e).  The  receptive  field  of  an  A-ll  amacrine  element  is  limited  to  bipolars  lying  on  one  axis  of  the  X-Y  plane  and 
behind  its  overlying  bipolar  relative  to  the  direction  selectivity  for  that  plane  of  elements.  The  bipolar's  influence  on  the  amacrine  ele¬ 
ments  is  subject  to  I/d.  where  d  is  the  horizontal  distance  between  them.  In  practice,  we  have  limited  d  to  the  value  of  3. 
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A,,Dij(t)  =  e  *  AIIDij(t  1)  +  2  ^(i  or  j)  ±  dKt)ou,/<l 
d=l 

where  D  is  the  direction  selectivity.  D  determines  which  index  (i  or  j)  to  increment  (+)  or  decrement  (-).  The  residual  inhibition  has  an 
exponential  decline  and  so  is  maximal  with  recent  movement  in  the  null  direction. 


Output  to  a  direction-sensitive  ganglion  element  is  the  bipolar  activity  that  exceeds  the  local  residual  inhibition  from  the  A-ll 
amacrine  associated  with  that  direction. 


AI,Dij(t)out  =  Pos(Bij(t)Out  -  AlIDij(t_  | )} 

In  the  variation  using  persistent  excitation,  type  A-II  amacrine  (All)  elements  also  accumulate  bipolar  output  with  a  persistence 
factor  of  e, 


A,,ij(t)  =  e  *  A,,ij(t-I)  +  Bij(t)out 

but,  the  horizontal  spread  of  All  potentials  creates  a  facilitatory  gate  for  the  transmission  of  new  bipolar  activity  on  to  the  ganglion 
element.  The  sensitivity  of  this  gate  is  itself  modulated  by  a  portion  (f)  of  the  sector's  A-l  amacrine  activity. 

A,IDij(t) out  *  <AII((i  or  j)  ±  I|(t-I)  *  Bij(t)out> '  <f  *  AI> 

A  recent  history  of  activity  from  bipolar  elements  on  one  side  biases  the  amacrine  element's  sensitivity  to  activity  from  bipolar  elements 
on  the  other  side. 


Ganglion  Elements.  In  both  variations,  there  are  four  ganglion  (G)  elements  per  sector,  each  defining  a  preferred  direction  (D). 
Each  ganglion  element  integrates  activity  from  all  of  the  bipolar  elements  in  its  sector  with  mediation  by  the  A-II  amacrine  elements, 
and  compares  that  sum  with  a  portion  (h)  of  the  accumulated  potential  on  the  A-l  amacrine.  Lateral  inhibition  between  complemen¬ 
tary  ganglion  elements  of  a  sector  (left  vs.  right,  and  up  vs.  down)  prevent  contradictory  responses  even  in  conditions  of  high  noise. 


GD  =  2  A,,Dij  ow  -  2  A,,-Dijout 


Gjy  out 


I  if  GD  >  h  •  Al 
if  Gp  <  h  *  AI 
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The  activity  Mate  of  the  ganglion  element  it  set  to  I  if  the  ganglion  potential  exceeds  its  threshold  (the  ganglion  element  fires),  otherwise 
it  is  set  to  zero.  The  activity  states  of  the  ganglion  dements  code  the  network's  decisions  about  the  certainty  of  coherent  movement  in  a 
sector  of  the  visual  field.  Increasing  the  A-I  amacrine  element  potential  is  equivalent  to  raising  the  criterion  for  certainty.  Ganglion  ele¬ 
ment  activity  in  a  sector  provides  location  information  to  the  resolution  of  the  size  of  the  sector.  The  speed  erf  movement  is  coded  in  the 
pulse  frequency  of  the  ganglion  element  output.  The  direction  of  movement  in  a  sector  is  determinable  from  the  relative  firing  rates  of 
the  ganglion  elements  since  they  code  speed  of  movement  along  orthogonal  X  and  Y  axes. 

SIMULATION  HARDWARE 

The  model  was  implemented  in  the  C  programming  language.  Simulations  were  performed  on  a  Sun  Microsystems  workstation. 
An  IBM/PC-AT  was  used  with  a  frame-grabber  to  study  digitized  video  images.  The  1BM-PC/  AT  insured  compatibility  with  the 
I8M-PC/ AT  bus  used  by  the  laboratory's  free-swimming  underwater  vehicle  (EAVE-WEST).  The  field-of-view  of  the  video  camera 
was  approximately  32  degrees.  This  provided  2  degrees  of  visual  angle  between  the  receptors  using  a  16-  by  16-receptor  field.  The  dig¬ 
itized  frame  of  512  by  512  was  sampled  at  32-pixel  intervals  to  produce  the  smaller  array.  The  IBM-PC/  AT  update  rate  with  the 
1-variant  was  2.5  frames  per  second  while  that  of  the  F-variant  was  4  frames  per  second.  This  is  barely  adequate  for  most  practical  real¬ 
time  visual  work.  Comparable  simulations  on  the  Sun  ran  eight  times  as  fast. 

BEHAVIOR  OF  THE  MODEL 

Criteria  for  adequate  behavior  were  ( 1 1  rapid  fading  of  a  stationary  image,  (2)  increase  in  the  output  pulse  rate  proportional  to  the 
speed  of  image  movement,  (3)  increase  in  x/y  output  asymmetry  proportional  to  the  direction  of  image  movement  off  the  diagonal, 
and  (4)  unambiguous  response  to  movement  with  signal-to-noise  ratios  (S /  N)  of  5  or  better. 

The  two  variations  of  the  model  were  tested  using  the  parameters  given  in  Table  11. 


Table  II.  Parameters  Used  in  Test  of  Model. 


Parameter 

1-variant 

F-variant 

Amacrine  I  Persistence  = 

0.50 

050 

Amacrine  II  Persistence  = 

0.70 

0.50 

Amacrine  1  to  Ganglion  Gain  = 

0  10 

0.10 

Amacrine  I  to  Amacrine  II  Gain  - 

— 

0.10 

Insensitivity  to  Stationary  Images.  In  the  present  model,  the  horizontal  elements  guarantee  that  the  image  will  disappear  if  sta¬ 
tionary  for  two  or  more  program  cycles.  Because  the  bipolars  respond  to  the  difference  between  the  horizontals  and  receptors,  only 
positive  intensity  differences  are  passed  on  to  the  amaennes.  However,  the  model  generally  works  well  for  both  increases  and  decreases 
in  light  intensity  if  either  brightening  or  dimming  detection  is  used.  This  is  because  an  inverse  image  trail  accompanies  any  movement 
(except  an  expanding  light  region  with  dimming  detection  or  an  expanding  dark  region  with  brightening  detection).  The  motion  sensi¬ 
tivity  of  the  model  is  similar  to  the  performance  of  the  artificial  retina  developed  by  Mead  and  Mahowald  ^S  They  were  able  to  imple¬ 
ment  in  VLSI  the  first  three  types  of  retinal  cells.  Our  model  extends  this  processing  through  to  the  ganglion  cell  layer  providing  infor¬ 
mation  on  speed  and  direction  of  movement. 

Speed  Sensitivity.  In  the  absence  of  noise,  the  model  reliably  produced  an  output  each  time  the  image  moved  from  one  receptor 
to  another.  Slowly  moving  images  simply  persisted  on  receptors  and  horizontals  and  disappeared  from  bipolars  until  a  new  receptor 
was  crossed  and  stimulated  by  a  change  in  brightness.  The  integrated  output  of  the  ganglion  elements  was  related  to  the  speed  of  image 
movement  between  two  limits.  The  lower  limit  resulted  from  the  loss  of  A-U  activity  that  blocked  or  gated  the  bipolar  output  to  the 
ganglion  layer,  governed  by  the  A-ll  persistence.  The  upper  limit  was  due  to  the  sampling  rate  of  the  system.  The  l-variant  was  more 
successful  at  following  slowly  moving  images  («  0. 1  degree  of  visual  angle  per  second)  because  A-II  persistence  could  be  set  at  a 
much  higher  level  than  in  the  F-variant.  Under  high-noise  conditions,  the  residual  excitation  of  the  F-variant  acted  like  an  indiscrimi¬ 
nate  sieve  that  passed  most  activity  on  through  to  the  ganglion  layer. 

Direction  Selectivity.  Images  that  were  moved  on  either  the  horizontal  or  vertical  axes  generated  unambiguous  responses  from 
detectors  aligned  with  the  movement.  Off-axis  movement  resulted  in  appropriate  combinations  of  vertical  and  horizontal  responses. 

The  mechanism  responsible  for  this  effect  was  the  relative  number  of  receptors  crossed  by  the  moving  image  per  unit  of  time.  Under 
some  conditions,  however,  the  direction  of  movement  defined  by  the  relative  ganglion  output  was  orthogonal  to  the  orientation  of  the 
leading  edge  of  the  image  and  bore  no  relationship  to  the  true  direction  of  movement  of  the  center  of  mass  of  the  moving  object.  The 
conditions  under  which  this  occurred,  such  as  when  the  object  was  much  larger  than  the  visual  field,  prevented  the  determination  of  the 
objectls  center  of  gravity. 

Noise  Immunity.  The  effects  of  noise  on  the  ability  of  the  retinal  model  to  track  a  moving  image  were  explored  by  varying  the 
SI  N  in  simulations.  The  S/  N  was  defined  by  the  ratio  of  the  com  rest  difference  between  the  moving  image  and  the  mean  background 
intensity,  and  the  rats  of  the  background  at  each  sample.  Background  noise  was  random  in  both  temporal  and  spatial  domains  but  its 
variance  was  consistent  over  time.  Figure  5  provides  the  error  rate  as  a  function  of  S/  N  for  the  two  variations  of  the  model.  An  error 
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Figure  5.  Errors  in  ganglion  cell  output  as  a  function  of  noise. 


was  defined  as  either  an  omission  of  the  appropnate  response  or  as  the  commission  of  an  inappropnate  response.  The  target  was  a 
homogeneous  line  extending  the  full  width  of  the  visual  field  and  moving  at  the  maximum  velocity.  The  error  curve  describes  a  sig¬ 
moidal  relationship  to  S / N.  In  the  l-variant.  errors  were  negligible  until  the  S;  N  was  reduced  to  4.  Then  the  error  rate  accelerated  to 
asymptote  at  75%.  In  the  F-variant.  the  error  curve  demonstrated  greater  accuracy  than  the  1-variant  with  higher  levels  of  noise.  The 
F-vanant  was  95%  accurate  at  a  S  N  of  3.  The  maximum  error  rate  of  75%  was  reached  by  both  variations  of  the  model  at  a  S  N  near 
I  Random  firings  of  the  4  ganglion  elements  would  guarantee  the  maximum  error  rate. 

THE  CONTEXT  OF  VISUAL  MOTION  ANALYSIS 

The  success  of  our  model  must  be  assessed  by  its  ability  to  extract  relevant  information  from  the  environment  This  information 
will  be  used  by  additional  processing  stations  to  generate  an  appropriate  motor  response.  Through  simple  reflex  connections  between 
movement  detectors  and  motor  controllers,  the  vehicle  could  seek  out  and  orient  to  man-made  objects  such  as  anchor  chains  or  pilings. 
Collisions  with  either  stationary  or  moving  objects  could  be  avoided  using  the  same  information.  The  reflexes  that  steer  a  vehicle 
toward  some  feature  in  the  visual  field  could  be  inverted  to  steer  the  vehicle  away  from  the  feature.  Additional  processing  stations 
would  be  tasked  with  making  the  decision  to  approach  or  to  avoid.  One  criterion  for  such  a  decision  may  be  the  velocity  of  closure. 

Image  Movement  Caused  by  Motion  of  Vehicle  Movement  of  the  vehicle  in  a  stationary  environment  will  cause  all  of  the  fea¬ 
tures  to  move  on  the  receptor  surface.  If  the  vehicle  is  moving  horizontally  relative  to  a  vertical  line,  a  large  response  will  be  generated 
in  the  vertical  detectors,  while  vertical  movement  of  the  vehicle  against  the  same  vertical  line  will  allow  it  to  go  v  irtually  undetected 
Appropriately  connected  to  motor  controllers,  the  detector  output  wilt  tend  to  keep  the  most  significant  features  in  the  center  of  the 
visual  field.  This  response  is  similar  by  design  to  the  optokinetic  reflex  of  insects  that  keeps  them  moving  in  straight  lines  and  compels 
them  to  orient  to  high  contrast  edges.^  This  behavior  is  extremely  useful  for  underwater  tasks  such  as  cable  tracking. 

Image  Movement  Caused  by  Motions  of  Both  Object  and  Vehicle.  Independent  movement  of  objects  in  the  visual  field  of  a 
moving  vehicle  can  occur  in  a  tracking  scenario.  If  there  are  no  stationary  objects,  then  the  problem  of  tracking  is  simplified  to  that  of 
orienting  to  a  moving  target  from  a  stationary  platform,  or  to  a  stationary  target  from  a  moving  platform,  for  either  condition  will 
result  in  similar  eff-cts  in  the  movement  analysis  network.  However,  if  both  the  vehicle  and  the  target  are  moving  in  a  field  of  station¬ 
ary  objects,  some  method  must  be  applied  to  filter  the  motion  information  due  to  the  vehicle's  movement.  Since  collision  avoidance  will 
likely  be  required,  this  information  cannot  simply  be  discarded.  Rather,  the  unique  motions  of  the  target  must  be  identified.  The 
amphibian  appears  to  sidestep  this  problem  by  remaining  motionless,  or,  if  the  environment  persists  in  moving,  by  stabilizing  the  back¬ 
ground  movement  with  an  opposing  movement  of  the  eyes.??  An  alternative  to  the  amphibian  strategy  is  to  have  a  secondary  motion 
processing  station  respond  to  the  change  in  detector  output  such  that  the  continuous  motion  information  can  be  attributed  to  vehicle 
movement  while  deviations  limited  to  some  sectors  could  be  attributed  to  target  movement.  For  example,  decelerations  in  all  sectors 
indicate  that  the  vehicle  itself  is  slowing  down,  while  a  deceleration  in  one  sector  indicates  that  a  target  has  moved.  Information  from 
other  sensors  such  as  Doppler  sonar,  accelerometer,  ir  altimeter  can  also  be  used  to  assess  vehicle  movement  and  assist  in  visual 
fixation*. 
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Nojse  Sensitivity.  The  test  of  noise  sensitivity  that  we  used  simulates  a  noise  condition  in  which  contrast  is  continually  changing 
in  all  directions  without  consistent  pattern.  As  a  well-defined  image  moves  among  these  apparitions,  it  is  degraded  by  loss  of  back¬ 
ground  contrast,  and  by  feed-forward  inhibition  from  activity  in  advance  of  its  movement.  A  situation  in  nature  comparable  to  this 
noise  condition  is  active  clutter.  Backscatter  in  the  underwater  environment  caused  by  suspended  particles  may  generate  similar  condi¬ 
tions,  but  will  tend  to  be  more  homogeneous  or  diffuse.  The  problem  of  clutter  for  a  motion  analysis  system  is  the  reason  tennis  is  not 
played  with  more  than  one  ball.  Obviously  the  problem  is  difficult  to  solve  even  with  all  of  the  visual  capability  of  man. 

Velocity  Sensitivity.  In  the  absence  of  noise,  the  present  model  codes  velocity  of  an  object  moving  across  the  visual  field  by  the 
■  frequency  of  action  potentials  generated  at  the  ganglion  elements.  The  maximum  velocity  to  which  the  model  can  encode  is  limited  by 

the  program  cycle  (or  update)  time  when  running  on  a  serial  machine.  This  is  currently  four  frames  per  second  on  an  IBM-PC/  AT. 
With  the  16-  by  16-receptor  array  and  a  32-degree  field-of-view,  this  translates  to  8  degrees  of  visual  angle  per  second.  For  comparison, 
the  maximum  velocity  that  can  be  tracked  by  a  cell  in  the  visual  cortex  is  about  2$  degrees  visual  angle  per  second.^  ]n  an  imple¬ 
mentation  where  it  is  possible  to  process  the  receptor  input  in  parallel,  output  will  still  be  proportional  to  velocity,  but  the  maximum 
encodable  velocity  will  be  limited  only  by  the  throughput  speed  of  the  system. 

At  velocities  below  the  maximum  for  the  model,  the  output  over  time  is  proportional  to  the  velocity.  The  system  tracks  slow 
moving  objects  in  the  absence  of  noise  down  to  velocities  in  the  range  of  0. 1  degree  visual  angle  per  second.  For  comparison,  the  lower 
limit  of  a  cortical  cell  to  track  a  moving  image  is  about  1  minute  of  visual  angle  per  second.^* 

BIOLOGICAL  FIDELITY  OF  THE  MODEL 


Of  the  two  variations  of  the  model  that  we  have  described,  the  I-variant  is  more  aligned  with  known  biological  mechanisms  than 
the  F-Variant.  Several  studies  have  shown  that  inhibition  is  necessary  for  the  null  response  from  direction-sensitive  ganglion  cells. 32,18 
The  F-Variant  may  model  the  effects  of  thalamic  efferent  input  to  the  retina.  The  facilitation  due  to  excitation  moving  in  advance  of 
the  image  improved  the  model's  recognition  of  image  direction  with  noise.  Both  models  deviate  from  biology  by  oversimplifying  and 
omitting  many  details  of  retinal  function  such  as  rod  adaptation,  rod  convergence,  and  some  aspects  of  bipolar  to  amacrine  and  ama- 
crine  to  ganglion  mapping. 

SUMMARY 

We  have  described  two  variations  on  a  very  simple  method  of  machine  motion  analysis  based  on  a  partial  model  of  the  verte¬ 
brate  retina.  One  variation  involves  lateral  inhibition  in  the  null  direction,  while  the  other  uses  lateral  facilitation  in  the  preferred  direc¬ 
tion.  The  model  produces  an  output  proportional  to  the  speed  of  image  movement  along  the  X  and  Y  axes  of  the  receptive  field.  Both 
direction  and  velocity  are  available  from  this  information.  We  have  noted  digressions  of  our  model  from  available  neuro-physiological 
data,  but  offer  the  model  as  a  working  solution  to  the  problem  of  visual  motion  information  for  guidance  of  autonomous  vehicles, 
rather  than  as  a  hypothesis  of  biological  mechanisms.  We  do  suggest  that  the  number  of  parsimonious  solutions  is  limited,  and  because 
nature  tends  to  exploit  simple  solutions,  some  insight  into  unknown  biological  mechanisms  may  be  gained  through  this  type  of 
exploration. 


CONCLUSIONS 

The  development  of  a  complete  ANS  is  an  ambitious  objective  of  science  fiction  proportions.  Even  with  a  century  of  neurobio- 
logical  information  waiting  to  be  applied,  and  the  neurosciences  experiencing  a  heyday  of  discovery,  the  knowledge  gaps  are  enormous. 
Reverse  engineering  of  the  brain  will  still  require  a  lot  of  creativity.  The  prospects  for  success  are  good,  however,  if  we  start  with  very 
simple  systems,  where  much  is  known,  and  where  the  behaviors  or  performance  criteria  are  well  defined.  Yet,  because  of  the  simple  per¬ 
formance  capability  of  these  reflex  systems,  a  degree  of  patience  must  be  exercised  early  on  until  sufficient  complexity  and  sophistica¬ 
tion  is  achieved  to  solve  hard  problems. 

We  have  demonstrated  this  approach  with  the  model  of  motion  analysis  in  the  vertebrate  retina.  This  provides  us  with  a  basis 
upon  which  to  move  into  the  brain  and  construct  an  artificial  visual  system.  But  we  are  not  just  studying  vision;  rather  we  are  trying  to 
understand  vision  in  the  context  of  sensory/  motor  integration.  A  larger  and  more  ecological  perspective  would  have  us  consider  vision 
in  the  context  of  the  vehicle’s  survival  in  the  ocean  environment. 

The  limitations  of  software  based  simulations  are  obvious.  Attention  should  be  given  to  the  rapid  implementation  of  these 
designs  in  VLSI  or  other  media  that  will  sustain  the  highly  parallel  and  adaptive  processing.  Reflex  systems  are  ideal  for  VLSI  because 
they  do  not  generally  require  long-term  modifications  to  the  connection  strengths,  but  do  require  their  modulation.  In  the  beginning,  a 
feasible  ANS  can  be  envisioned  with  a  large  number  of  chip-resident  components  communicating  with  and  supervised  by  a  hierarchy  of 
distributed  processing  machines  of  more  conventional  design.  The  early  application  of  ANS  will  allow  a  survival-of-the-fittest  testing 
under  real-world  conditions  and  further  the  evolutionary  development  of  this  new  kind  of  intelligent  species,  the  AUV. 
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